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Abstract 

The main contributions of this paper are three fold. First, our primary concern is to investigate a 
class of stochastic recursive delayed control problems which arise naturally with sound backgrounds but 
have not been well-studied yet. For illustration purpose, some concrete examples are also provided here. 
We derive the stochastic maximum principle of sufficient condition to the optimal control in both cases 
with and without control delay. Second, it is interesting that a new class of time-advanced stochastic 
differential equations (ASDEs) is introduced as the adjoint process via duality relation. To our best 
knowledge, such equations have never been discussed in literature although they possess more academic 
values besides the control study here. Some existence and uniqueness result to ASDEs is presented. 
Third, to illustrate our theoretical results, some dynamic optimization problems are discussed based on 
our stochastic maximum principles. In particular, the optimal controls are derived explicitly by solving 
the associated time-advanced ordinary differential equation (AODE), the counterpart of the ASDE in its 
deterministic setup. 

Key words: Advanced stochastic differential equation (ASDE), Backward stochastic differential equation 
(BSDE), Maximum principle, Pension fund with delayed surplus, Stochastic recursive delayed control. 



1 Introduction 

Our starting point is the following backward stochastic differential equation (BSDE) with time-delayed 
generator: 

' -dy{t) = fit, y{t), y{t - 6), z{t), z{t - 6))dt - z{t)dW{t), t G [0, T], 
< y{T)=^, (1) 
^ y{t)=ip{t), z{t)=i^{t), te[-6,0). 
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Two remarkable features of Eq.©: (i) The terminal instead initial condition is specified; (ii) The generator 
/ depends not only on the instantaneous state {y{t), z{t)) but also on {y{t — 6), z{t — 6)) through the time- 
delayed parameter 6 > 0. The feature (i) makes Eq.© essentially different to the well-studied stochastic 
delay differential equation (SDDE) (see e.g. Mohammed |[T6l . |[T7l . etc.) in which the initial state condition 
is given beforehand. Eq.([T]) also differs from the standard BSDE due to its time-delayed generator from 
(ii). In particular, it distinguishes from the anticipated backward stochastic differential equation (ABSDE) 
introduced by Peng and Yang ll22l which is the duality of SDDE. Eq.([Tll is first introduced by Delong 
and Imkeller Q and it has many real backgrounds in economics, finance, management, or other decision 
sciences. More details can be found in Delong Q, lH, Delong and Imkeller Q and the reference therein. 
Due to the interesting structure and wide-range applications, it is very natural and necessary to study the 
dynamic optimizations of Eq.([T])- However, to our best knowledge, very few works have been done along 
this direction thus we aim to fill this research gap in some systematic way. To this end, we study the 
following more general controlled backward delayed system: 

-dyit) = f it,y{t), (j){t,s)y{s)a{ds),z{t), (j){t, s)z{s)a{ds),v{t), (pit, s)v{s)a{ds)] dt 



Its Jt-5 Jt-5 

-z{t)dW{t), te[0,T], 

y{T) = 

y{t)=ip{t), z{t)=i^{t), te[-6,0). 

(2) 

Here, 6 is time delay parameter, a is some (T-finite measure and (/){■, •) is some bounded process. The rele- 
vance and importance of our optimization problems can be illustrated by the following concrete examples. 

Example 1.1 {Optimization of recursive utility with moving average) This example originates from Delong 
151 in which the decision makers have recursive utility with moving average generators. Such utility can be 
used to characterize the habit information, disappointment effects as well as volatility aversion in decision- 
making. Accordingly, the objective of decision maker is to maximum his/her utility by selecting suitable 
instantaneous consumption process c(t). This leads to the following dynamic optimization problem 

inf y^(0) 

where the recursive utility y{t) satisfies the following BSDE with time-delayed generator 

-dy{t) = f (^t, y{t), ^ ^ y{s)ds, z{t), ^ ^ z{s)ds, c{t)^ dt 

-zit)dWit), te[0,T], (3) 

y{t)=^{t), z{t) = i;{t), te[-S,0). 

Eq.^ can be viewed as the special case of Eq.© by noting j y{s)ds = J^_j, y(s) jX{s>o}a(ds) where 
a is uniform measure on [t — T,t\. It can characterize the non-monotonic utility to volatility aversion. 
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Example 1.2 (Pension fund with delayed surplus) This example comes from Federico lH where the pen- 
sion fund manager can invest two assets: the riskless asset Po{t) satisfies dPQ{t) = rPQ{t)dt with instanta- 
neous return rate r > 0, and the risky asset Pi{t) satisfies 

dPi{t) = nPi{t)dt + aPi{t)dW{t), 

with return rate fj- > r, volatility rate a > 0. Here, W{-) is a. standard Brownian motion. Denote A = 
the risk premium, 6{t) G [0, f ] the proportion of fund invested in risky asset, and S{t) the surplus premium 
to fund members. Suppose the wealth of pension fund at time t is y{t), and it is reasonable to assume S{t) 
depends on the performance of fund growth during the past period. Thus, we assume: 

Sit)=g{y{t)-Kyit-S)) 

for some k > and g( : M — )■ [0, +oo) which is increasing, convex and Lipschitz continuous, 5 > is the 
time delay. On the other hand, there should be some running cost or consumption for fund management, 
which is represented by the instantaneous rate c{t). Hence the wealth process y{t) evolves as: 

' dy{t) = ([0(t)aA + r]y{t) - g{y{t) - Ky{t - S)) - c{t)) dt + ae(t)y{t)dW [t), t G [0, T], 

< y(0)=yo, (4) 
^ y(t)=0, tG[-5,0). 

Note that in practice, the pension fund will be required to provide some minimum guarantee, i.e., to pay 
some part of the due benefits ^ (which is some random variable) at some given future time T. Keep this in 
mind, the objective of fund manager is to choose 9{t) and c{t) to reach terminal condition y{T) = ^, and 
also maximize some given cost functional at the same time. By setting z{t) = a9{t)y{t), Eq.^ can be 
reformulated by the following controlled backward delayed system: 

' dy{t) = {ry{t) + Xz{t) - g{y{t) - Ky{t - 5)) - c{t)}dt + z{t)dW{t), t G [0, T], 
< y(t)=0, tG[-(^,0), (5) 

. y{T) = e 

Eq.® is a special case of ^ by setting a{ds) to be Dirac measure at —5, the pointwise delay with lag 5. 

Example 1.3 It is remarkable that there exist considerably rich literature to discuss the controlled stochas- 
tic delay differential equations (SDDEs) (see e.g. ||2]|, ifTTI . |[T9l etc.) which arise naturally due to the time 
lag between the observation and regulator, or the possible aftereffect of control. The SDDEs and its opti- 
mization have attracted extensive research attention in last few decades, and have been applied in wide-range 
domains including physics, biology and engineering, etc. (see l|T6l . lITTl for more details). Note that these 
works are discussed in the forward setup because the initial condition is given as the priori. On the other 
hand, as suggested by Kohlmann and Zhou |[T4l . Ma and Yong ifTSl . the forward controlled systems can 
be reformulated into some backward controlled systems under mild conditions. For example, in case of 
some state constraints (e.g. no short selling), it is better to reformulate the controlled forward systems into 
some backward systems which are more convenient to be analyzed in some cases (see Ji and Zhou fTO\, El. 
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Karoui, Peng and Quenez lIlTTl '). Also, inspired by Lim and Zhou lfT2]| . we aim to investigate the following 
controlled linear backward delayed system: 



dy{t) = {Piy{t) + ^2y{t - 6) + -fiz{t) + -f2z{t - 6) + av{t)) dt + z{t)dW{t), te [0,T], 
y{T)=i, tG[-<5,0], 



which can be viewed as the linear constrained forward controlled delay system by using penalty approach, 
or the limit of a family of linear unconstrained forward delayed system. 

The rest of this paper is organized as follows. In Section 2, we introduce the advanced stochastic differ- 
ential equation (ASDE). Some preliminary results on ASDE and the associated BSDE with delay generator 
are also given. The stochastic recursive delayed control problems are formulated in Section 3, and two max- 
imum principles are derived based on the duality between the ASDE and the BSDE with delayed generator. 
As the application of our theoretical results, in Section 4 — 6 we revisit some motivating examples given in 
Section 1 and the optimal controls are derived explicitly by solving the associated time-advanced ordinary 
differential equation (AODE). 

2 Notations and Preliminaries 

Let T > be some finite time horizon. For any Euclidean space H, we denote by (•, •) (resp. | • |) the scalar 
product (resp. norm) of H. Let M"^™ be the Hilbert space of all n x m matrices with the inner product 



Here the superscript T denotes the transpose of vector or matrix. Let W{-) be a standard (i-dimensional 
Brownian motion on a complete probability space (17, J^, P). The information structure is given by the 
filtration F = {Tt}t>Q which is generated by W{-) and augmented by all P-null sets. For p > 1, the 
following notations are used throughout this paper: 

i^(r2, J^i , P; H) := is H-valued J^t — measurable random variable satisfying E[^''] < +oo}; 
X^(ti, t2; H) := ti < t < i2, is F — adapted process satisfying E J"^*^ \Lp{t)\'Pdt < +oo}, 

L|P(ti, ^2; H) := {^{t), ti < t < t2, is H-valuedF — adapted bounded process}. 



{A,B) ■.= tr{AB^}, \/A,B£W 



nxm 



We set 





Then the backward delayed system of form ([2]) can be rewritten as 



dy{t) 

y{T) 
y{t) 



f{t,y{t),ys{t),z{t),zs{t))dt-z{t)dW{t), t e [0,T] 
ip{t), z{t) = i;{t), t£[-6,0). 



< 



(6) 



We introduce the following assumptions: 
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(H2.1) The function f : Q x [0,T] x W x W x M"^'^ x W'^'^ W is F-adapted and satisfies 

\f{t,y,ys,z,zs) - f{t,y',ys,z',z's)\ < C{\y - y'\ + \ys - y'sl + \z - z'\ + \zs - 41) 

for any y, ys, y', y'^ G M", z, zs, z', 4 ^ M"^'^ with constant C > 0. 

(H2.2) The fixed time delay satisfies < 5 < T, ^ e L2(J7, J't, P; M"), the initial path of {y,z): 
ip{-),Tp{-) are given square-integrable functions and 4>(t,s) < M is given bounded J-'^— adapted process 
with < s < t < T and M is some positive constant. 

(H2.3)E[/(f 0,0,0, 0)|2dt] < +00. 

Then we have the following existence and uniqueness of the delayed BSDE Q: 

Theorem 2.1. Suppose that (H2.1)-(H2.3) hold, then for sufficiently small time delay 6, the BSDE with 
delay ^ has a unique adapted solution {y{-),z{-)) G L|(— 5, r;M") x L|(— (5, T; M"^'^). 

Proof Let us introduce the following norm in Banach space 5, T; M") which is equivalent to the 
original norm of L|(-(5, T; M"): 

||K-)ll/3 = (E[|jz^(.)|V^d.])i 

Set ^ ^ 

y{t)=i+f f{s,Y{s),Ys{s),Z{s),Zs{s))ds- [ z{s)dW{s), t e [0,T], 

Jt Jt (7) 

^y{t)=ip{t), z{t)=iP{t),te[-S,0). 

Define a mapping h : Ll{-5, T; x W"^) — ^ M" x M'^^'^ such that h[{Y{-), Z(-))] = iy{-), z{-)). So if 
we can prove that his a. contraction mapping under the norm || • ||/3, then the desired result can be obtained by 
the fixed point theorem. For two arbitrary elements {Y{-),Z{-)) and {Y'{-), Z'{-)) in Lj{-6,T;W xW''^), 
set {y{-),z{-)) = h[{Y{-), Z{-))] and (y'(-), z'{-)) = (•), Z'{-))]. Denote their difference by 

i(-)) = {Y{.) - y (•), Z{-) - Z'{.)), (y(.), z{-)) = (y(.) - y'{-),z{.) - z'{.)). 

In fact Eq. ([7]) is a classical BSDE, and it follows that 

nj\^\y{s)? + \z{s)?)e^'ds] 

< |e[ r \fis, Yis), Ysis), Zis), Zs{s)) - fis, Y'is),Y^is), Z'{s), Z'sis))\^e^^ds] 
P Jo 

i2 f-T 



< {\y{^)\ + \ys{s)\ + \zis)\ + \zs{s)\fe^^ds] 

an2 f-T 

< — E[ / + \Z{s)\^ + 2|y5(s)|2 + 2\Zs{s)\^)e^'ds] 



fi JO 

a(-i2 fO f-T 

<-^[l + 2M^6 e-^''aidr)]E[ {\Y{s)\'^ + \Z{s)\'^)e^'ds] 
P Js J-s 

= KiC,M,5,a,m r {\Y{s)\^ + \Z{s)\'')e^'ds]. 
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Note that 



E / \Ys{s)\'^e^'ds 



Jo 



= E / 1/ (f>{s,s + r){Y{s + r)-Y'{s + r))a{dr)\'^ef^'ds 



< M^6E / / \Y{s + r) - Y'{s + r)\'^a{dr)e^'ds 




J-5 
[-0 r-T 



= M^5E e-l^" \Y{s + r)-Y'{s + r)\^e^^'+''^dsa{dr) 



J-5 Jo 
pO pT+r 



= M^5E e-^'- \Y{u)-Y'{u)\^ef^''dua{dr) 




<M^6E e'l^''a{dr) \Y{s)\^el^'ds. 
J-5 J-5 

If we choose /? = j, then 

K{C, M, 6, a, /?) = 6C^d[l + 2M'^6ea{[-6, 0])]. 

Therefore, if 6 is sufficiently small satisfying K{C, M, 6, a, /?) < 1, then /i is a contraction mapping under 
the norm || • ||^. Our proof is completed. 

Now, let us introduce the following advanced SDE as following: 

dx{t) = b (t, x{t), I (j){t, s)x{s)a{ds)) dt + a (t, x{t), [ (j){t, s)x{s)a{ds)) dW{t), t G [0, T], 



It is notable that there exist some results to discuss the time-advanced ordinary differential equations (AODEs) 
(e.g., refer |[T1, 111, fl3l . ifTSl . Il20l . Il24l . etc.) which have been applied in various areas including travel- 
ing waves in physics, cell-growth in population dynamics, capital market in economics, life-cycle models, 
electronics, etc. However, to our best knowledge, the stochastic differential equations of advanced type 
(ASDE) has never been discussed before. Nevertheless, these stochastic advanced equations should also 
have considerable real meanings besides the control study only (as implied by the broad-range application 
of AODES, their deterministic counterpart). Keep this in mind, we will discuss these meanings in future 
study. Now we aim to study the /"t-adapted solution x(-) e L|(0, r + (5; W) of the ASDE dU). Suppose that 
for alH G [0,T], b : Q x W x L^{n,Tr,F;W) ^^(f^, jr^, P; M"), a : Q x W x L2(0, 7;, P; R") ^ 
L2(fi,J-i,P;R"X'^), where r G [t,T + 6]. We also assume that b and a satisfies the following conditions: 

(H2.4) There exists a constant C > 0, such that for alH G [0,T], x,x' G M", C(-); C'(-) e L|(t,T + 
5;R"), re[t,T + 6], we have 



t 



< 



x(0) = Xq, 

x{t) = \{t), t G {T,T + S]. 



(8) 



\b{t, X, C(r)) - b{t, x', C'(r))[ + \a{t, x, C(r)) - a{t, x' , C'(r)) 
<C(|x-x'|+E^'[|C(r)-C'(r)j]). 
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(H2.5) 



sup (|6(t,0,0) 0,0)1) < +00. 

0<t<T 



Under these conditions, b{t, •, •) and a{t, ■,■) are /"^ -measurable and this ensures the solution of the advanced 
SDE will be J^-adapted. We have the following result to the ASDE dHll. 

Theorem 2.2. Assume b and a satisfy (H2.4) and (H2.5), E|xoP < +00, Ksup2'<t<T+5 1-^(0 P < +00, 
and the time delay 5 is sufficiently small, then the ASDE dS]) admits a unique Tt-adapted solution. 

Proof Similar to Theorem l2.1[ let us define the following norm in Banach space L^(0, T+5; MP") which 
is more convenient for us to construct a contraction mapping: 

rT+S 



IK- 



1/3 



El 



\u{s)\^e-^'ds\Y. 



For simplicity, we denote f^^^ (j)(t, s)x{s)a{ds) by xs+ (t), and set 

x{t) = xo+ [\is,X{s),Xs+{s))ds+ f a{s,X{s),Xs+{s))dW{s),te[0,T], 
< Jo Jo 

^x{t) =X{t), te {T,T + 6]. 

Then we can define a mapping / : L|(0,r + 5;R") L|(0,r + 5;R") such that I[X{-)] = x(-). For 
arbitrary X{-),X'{-) £ L|(0, T + 5; M"), we introduce the following notations: 

I[X{.)]=x{.), I[X'{-)]=x'(:), 
X{-) = X{.)-X'{.), f(.)=x(.)-x'(.). 



Consequently, x(-) satisfies 



xit)= / [bis,X{s),Xs+is))-b{s,X\s),X's+{s))]ds 
Jo 

+ [ [a{s,X{s),Xs+is)) - a{s,X'is),X's+is))]dW{s), t e [0,T], 
Jo 

x(0) = 0, 
t x{t)=0, t G {T,T + 6]. 

Applying Ito's formula to e~^^\x{t)\'^ on [0, T], we get 

T 



E[e-^^|x(r)|2] + /3E[ / e-'^'\x{t)\^dt] 
Jo 



E| 



i2e-f''m,xit)) + e-^\ait),ait)))dt], 



with 



b{t) = b{t, X{t),Xs+ (t)) - b{t, X'{t), (t)), 
a{t) = a{t, X{t),Xs+ (t)) - a{t, X'{t),X'^+ {t)). 
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Since b, a satisfy (H2.4), we have 

rT 



/3E[ / e-'^'\x{t)\^dt] 
Jo 

<E[f e-^*|x(i)pdt]+E[/ e-^^\b{t)\^dt]+E[f e-^*\ait)\^dt], 
Jo Jo Jo 

<E[f e-^^\x{t)\'^dt] + 2C'^E\ f e-^WX{t)\+E^*[\Xs+{t)\]fdt 
Jo '-Jo 



Moreover, it follows that 



due to the fact 



Set 



l)E[/^e-^*|x(i)pdi] 
Jo 

<4C'^E[f e-'^'\Xit)fdt] + AC^E[[ e-(^'\Xs+{t)\'^dt] 
Jo Jo 

fS rT+S 

<4C^[1 + M^S e^'a{ds)]E[ e-^*|X(t)pdt], 
Jo Jo 

ElT e-^'\XsAt)M 
Jo 

= E[ e-^*| / (f>{t,s)X{s)a{ds)\'^dt] 
Jo Jt 

<M^SE[[ e"^* / \X{s)fa{ds)dt] 
Jo Jo 

S T 

= MHE[[ e^' f e-^^^+'\x{s)\^dta{ds)] 
Jo Jo 

1-5 fT+5 
<MH e^'aids)E[ e-^'\Xit)\'^dt]. 
Jo Jo 

4C^[1 + M^5j^e^'aids)] 



K'{C, M, 5,a,P) 



P-1 

If we choose P = then for sufficiently small 6, we have K'{C, M, 6, a, P) < ^c^S[i+My^ea{[o,S])] ^ 
follows the mapping / is contraction, hence the result. 

3 Optimal control problem for backward stochastic system with delay 

In this section we study a kind of stochastic recursive delayed control problems as follows: 

-dy{t) = f it,y(t), 4>(t,s)y(s)a{ds),z{t), 4>(t, s)z(s)a{ds),v{t), (f>{t, s)v(s)a{ds)] dt 



t-5 Jt-5 Jt-S 

-z{t)dW{t), te[0,T], 

[ y(t)=^{t),z{t) = i^(t), te[-5,Q). 



(9) 



Here f : Q x [0,T] x M" x x M"^*^ x R"^^'^ x M*^ x M*^ — > is given measurable function, 
^ G JV', P; M"), (^(•) is deterministic function. v{-) is the control process with initial path rj. The 

stochastic recursive control problems is to find the optimal control to achieve a pre-given goal ^ at the 
terminal time T, and also maximize some given cost functional. Let [/ be a nonempty convex subset. We 
denote Li the set of all admissible control processes v{-) of the form 




7]{t), te [-5,0), 

v{t) GL|(0,r;M'=), v{t) G U, a.s., t G [0,r]. 



The objective is to maximize the following functional over U: 

J{v{-))=E[[ l(t,y{t), [ (j){t,s)y{s)a{ds),z{t), [ (j){t,s)z{s)a{ds),v{t), [ (j){t, s)v{s)a{ds)) dt 
Jo \ Jt~5 Jt-S Jt-S ) 

+ 7(y(o))]. 

For simplicity, denote (//__5 </>(t, s)y{s)a{ds), fl:_g (/>(t, s)z(s)a((is), //_^ (pit, s)v{s)a{ds)) by {ys(.t), zs{t),vs{t)) 
if no confusion occurs. 

(H3.1) / is continuously differentiable in {y, ys, z, zs,v, vs)- Moreover, the partial derivatives fy, fy^ , fz, fzs , fv 
and /t,^ of / with respect to {y,ys, z, zs,v,vs) are uniformly bounded. 

Then if v{-) is admissible control and assumption (H3.1) holds, then the delayed BSDE Q has a unique 
solution {y"{-),z''{-)) G L|(0, r+(^; M") xL|(0, r+(^; M"^'^) on [0, T+J] for sufficiently small < 5 <T. 

(H3.2) For each v{-) G U, l{-,y''{-),y^s{-), z^'i-), z^{-),v{-),vs{-)) G L^{0,T-R), I is differentiable to 
{y,ys, z, zs,v,vs), ^ is differentiable with respect to y, and all the derivatives are bounded. 

Define the Hamiltonian function i7 : [0, T] x M" x M" x R"^'^ x M'^ x R'' x ^ R by 

H{t,y,ys,z,zs,v,vs) = l{t,y,ys, z, zs,v,vs) - {f{t,y,ys,z,zs,v,vs),p). 
For each v(-) G U, the associated adjoint equation satisfies the following ASDE: 

' dp^it) = { - Hy{t,e^{t),v{t),vs{t),p^{t)) 

-^^'[fj Hy,is,Q-{s),vis),vsis),p%sms,t)x[o,T]is)ds]^}dt 
+ {-Hzit,Q''it),v{t),vsit),p''{t)) 

-^^'[fj Hz,{s,Q-{s)Ms)Ms)y{sms,t)xyo,T]{s)ds]'^]dW{t), i G [0,r], 

. p^(o)= -7.(y(o)), 

(10) 

with Q'"{t) = {y''{t),y^g{t), z''{t), ^(t)) and ^ is the Radon-Nikodym derivative. 

Remark 3.1. For a given admissible control v{-), Eq.UOi is an ASDE. By the virtue of the indicative function 
X[o,T]i^)' ^t is not necessary to give the value of p^{t) on (T, T + S\. Moreover, the ASDE diOl ) admits a 
unique solution under condition (H3.1) and (H3.2) due to Theorem \2.2\ 
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Now we can give the first main result of this paper in the following: 

Theorem 3.2. (Sufficient condition of optimality) Let (H3.1) and (H3.2) hold. Suppose for «(•) G U, 
(y(-), z{-)) is the corresponding trajectory and p{-) the corresponding solution of adjoint equation rfiOl ). If 
the following condition holds true: 



{H^{t,e{t),u{t),us{t),p{t)) 



(11) 

= max{Hy{t,@{t),u{t),us{t),p{t)) 

veil 

/t+s a(dt) 
H,,{s, eis),u{t),us{s),pis))<P{s, t)x[o,T] is)ds]^^,v), 

moreover, if H{t, y, ys, z, zs,v, vs,p{t)) is a concave function of {y, ys, z, zs, v, vs), and 7 is concave in y, 
then u{-) is an optimal control for our problem. 

Proof Choose a v{-) G U and let {y'"{-), z^{-)) be the corresponding solution of To simplify the 
notation, we also use e^(t) = (y^(t), and e(t) = {y{t),y5{t), z{t), zs{t)). Let 

/ = E[ / {l{t, y{t),ys{t), z{t), zsit),u{t),us{t)) - l{t, y''{t),yUt), z^{t), zUt),vit),vs{t))} dt] , 
Jo 

//= [7(y(0))-7(y"(0))]. 
We want to prove that 

Jiu{-)) - J{vi-)) = I + 11 >0. (12) 

Since 7 is concave on y, 

n > jy{y{0)V{y{0) - y^O)) = -p{0)^{y{0) - y^O)). 
Applying Ito's formula to {p{-),y{-) — y^i-)), we have 
p(0)T(y(0)-y'^(0)) 
=E / {p{t),f{t,Q{t),u{t),us{t)) - f{t,e^{t),v{t),vsmdt 



JO 

+ e/ {Hy{t,eit),u{t),usit),p{t)) 







rt+5 ut] (13) 

+ E-^* [ Hy,{s, e{s),u{s),us{s),p{s))<l){s, t)x[o,T] {s)ds]^, y{t) - y"{t))dt 

+ E [ {H,{t,e{t),u{t),us{t),p{t)) 
Jo 

i-t+S a(dt) 
+ E^* [ / H,,{s, e{s),u{s),us{s),p{sms, t)xio,T] («)^«]-^' ^(*) " ^'(*))^*- 
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On the other hand, 



/ = E r [H{t,Q{t),u{t),us{t),p{t)) - H{t,Q''{t),v{t),vs{t),pmdt 







+ E I {p{t)J{t,eit),u{t),usit))- f{t,e''it),vit),vsmdt. 







Since (0, v, vs) H{t, 0, v, vs,p{t)) is concave, we have 

I> -E r {Hy{t,e{t),u{t),usit),pit)),y^it) - y{t))dt 



JO 

-E [ {Hy^{t,e{t),u{t),us{t),p{t)),yUt) - ys{t))dt 
Jo 

-E [ {H,it,e{t),u{t),usit),p{t)),z"{t) - z{t))dt 
Jo 

-E [ {H,,{t,eit),u{t),us{t),p{t)),z^sit)-zs{t))dt 
Jo 

-E I {H^{t,e{t),u{t),us{t),p{t)),v{t) -u{t))dt 
Jo 

-E [ {H^^{t,Q{t),uit),usit),pit)),vs{t) - usit))dt 
Jo 

+ E [ {p{t),f{t,Q{t),u{t),us{t)) - f{t,Q''{t),v{t),vsmdt. 
Jo 



Moreover, we have 



E / {H^^{t,e{t),uit),usit),p{t)),vsit) - usit))dt 
Jo 

= E {Hy^{s,e{s),u{s),us{s),p{s)), (l){s,r){v{r) - u{r))a{dr))ds 

Jo Js-5 
i-T i-r+S 

= E j {E^^- H^g{s,Q{s),u{s),usis),p{s))(j){s,r)x[o,T]is)ds,v{r) - u{r))a{dr) 

= E (E^ny^ H,,,{sMs),u{s),U5{s),p{s))<l,{s,t)x[o,T\{s)ds]^Mt)-u{^^^^^ 
By the maximum condition ([TTI) . we can obtain 

E / {H^{t,Q{t),u{t),U5{t),p{t)),v{t)-u{t))dt 
Jo 



+ E [ {H^^{t,e{t),u{t),us{t),p{t)),vs{t) - us{t))dt 







0. 
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From (fT2ll-([T7l). it is easy to get 
J(n(.) - J{v{-)) 

> - E [ {Hy{t,e{t),u{t),usit),p{t)),y''{t) - y{t))dt 
Jo 

-E / {Hy,{t,e{t),u{t),us{t),p{t)),yUt) - ysit))dt 
Jo 

-E [ {H,{t,e{t),uit),usit),p{t)),z''{t) - z{t))dt 
Jo 

-E [ {H,,{t,eit),u{t),us{t),p{t)),z^sit)-zs{t))dt 
Jo 

+ e[ {Hy{t,eit),u{t),usit),p{t)) 
Jo 

+ E-^'[^ ^ Hy,{s,Q{s),uis),usis),pisms,t)x[o,T]is)ds]^,y^{t) - y{t)^^^ 

+ E [ {H,{t,e{t),u{t),us{t),p{t)) 
Jo 

rt+S a(dt) 
+ E^*[y^ i/,,(s,G(s),n(s),n5(s),p(s))<A(s,i)x[o,T](5)ds]-^,z^(0-^(t))^it 

= 0. 

So, we verify that J(u(-)) — J{v{-)) > for any v{-) G U, and it follows that n(-) is the optimal control. 

Corollary 3.3. If the a{dt) is the Dirac measure at —6, then the system involves pointwise delay, i.e. 
ysit) = y{t — 6),zs{t) = z{t — 5),vsit) = v{t — 5). In this case, the sufficient condition ofoptimality is 

H^it, Q{t),u{t),u{t - 5), pit)) + E-^* [H^, {t + 5, e(t + 6),u{t),uit + 5), pit + 5))] = 0, 
with adjoint equation 

' dp^t) = { - Hyit,e%t),vit),v{t - 6),p%t)) 

- E-^* [Hy^it + 5, G"(t + 5), f (t + d),v{t),p''{t + 5))]}dt 
{-H,it,@^it),vit),vit-6),p^t)) 

* -r- (18) 

-E^'[H,^{t + 5,e''{t + 6),v{t + 6),v{t),p''{t + 6))]}dW{t), t G [0,r], 
^^^(0)= -7,(y(0)), 

=0, te{T,T + 6], 

where Q^t) = {y\t),y- {t - 5), z^ {t), z^ {t - 5)). 

Now we consider the special case wherein the control variable involves no delay, to derive the corre- 
sponding maximum condition, we first introduce the following condition. 

(H3.3) For each G ZY, /(•, y^(-), 2/^(-), ^''(•), ^^CO) e (0, T; R), Hs differentiable on (y, y^, 25) 
and 7 is differentiable with respect to y, all derivatives are bounded. 
We have the following result: 
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Theorem 3.4. (The case without control delay) In case there has no control delay, that is, 



Suppose m(-) € U, (y(-), z{t)) is its corresponding trajectory andp{-) the corresponding solution of adjoint 
equation f liOl ). Let (H3.1), (H3.3) and the following condition holds true: 

H{t,@{t),u{t),p{t)) =maxH{t,e{t),v,p{t)), forallte [0,T], (19) 

with @{t) = {y{t),ys{t), z{t), zs{t)), moreover, suppose for each {t,y,ys, z, zs) € [0,T] x M" x M" x 
j^nxd ^ K"xrf^ H{t, y, ys, z, Zs) = max„g(/ H{t, y, ys, z, zs,v,p{t)) is a concave function of {y, ys, z, zs), 
and 7 is concave in y, then u{-) is an optimal control. 

Proof Similar to the proof of Theorem 13.21 we also choose arbitrary v{-) G U, and aim to prove 
J(n(-)) — J(v{-)) > 0. From the procedure of Theorem 13.21 we can see that 

Jiui-) - J{vi-)) 

>E / [H{t,@{t),u{t),p{t)) - H{t,e''{t),v{t),pmdt 



Jo 

e[ {Hy{t,e{t),u{t),p{t)) 







+ E^' [ / Hy, {s, e(s), n(s),p(s))0(s, t)x[o,T] - y%t))dt 



dt 



T 



E / {H,{t,eit),u{t),pit)) 





+ E^n^*^ H,si^,e{s),u{s),p{sms,t)xio,T]{s)ds]^,z{t) - z^t))dt. 

By the condition ([T9l ) and the definition of H, 

H{t,e{t),u{t),p{t)) - H{t,y,ys,z,zs,v,p{t)) ^^^^ 
>H{t,e{t))-Hit,y,ys,z,zs). 

Since {y,ys, z, zs) H{t,y,ys, z, zs) is concave for any given t € [0, T], it follows that there exists a 
supergradient ai{t),a2{t) G and bi{t), b2{t) G R"^'^ for H{t, y, ys, z, zs) at (y, ys, z, zs) (refer Chapter 
5, Section 23 in 1231), that is, for all (y, ys, z, zs), 

H{t,y,ys,z,zs)-H{t,@{t)) ^^^^ 
< {ai{t),y- y{t)) + {a2{t),ys - ys{t)) + {bi{t), z - z{t)) + (ftsW, ^5 - ^^(t)), 

Define 

'^{t,y,y5,z,zs) = H{t,y,ys,z,zs,u{t),p{t)) - H{t,e{t),u{t),p{t)) 

- {ai{t),y- y{t)) - {a2{t),ys - ysit)) 

- {bi{t), z - z{t)) - {b2{t), zs - zs{t)). 
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Obviously, T{t,y,ys, z, zs) < for all {y,ys,z,zs) and T{t,@{t)) = 0. It follows that T attains its 
maximum value at ysit), z{t), zs{t)). Consequently we have 

ry{t,e{t)) = o, ry,{t,@{t)) = o, 
r,(t,G(t)) = o, r,,(t,e(t)) = 0. 

These will lead to 

Hy{t,e{t),u{t),p{t)) =ai{t), Hy^{t,e{t),u{t),p{t)) = a2{t), 
H,{t,Q{t),u{t),p{t)) = bi{t), H,^{t,Q{t),u{t),p{t)) = b2{t). 

Combine (l22l) and note the arbitrariness of {y, ys, z, zs), we have 

Hit,e^it))-H{t,eit)) 

< {Hy{t,e{t),u{t),p{t)),y-{t) - y{t)) + {Hy,{t,Q{t),u{t),p{t)),yUt) - ys{t)) 
+ {H,{t,e{t),u{t),p{t)),z'^{t) - z{t)) + {H,,{t,e{t),u{t),p{t)),zUt) - zs{t)). 

Substitute the above result into (l20b . we obtain J(n(-)) — J{v{-)) > 0. 



4 Application I: Dynamic optimization of recursive utility with moving av- 
erage 

In this section, we investigate Example 1.1: the dynamic optimization of recursive utility with moving 
average, which is already given in Section 1. The state equation satisfies the following dynamics: 



yit) = e 



T 



ac(s) + /3 ( - / yiu)du 
s Jo 



ds 



z{s)dWs 



(23) 



where a,/3 > are some constants, and the control variable is consumption process c(-). The class of 
admissible controls is denoted by C = {c(-) € L|(0, r;M),t € [0,r]}. Given some standard utility 
function U, for example, U{x) = ^ for < i? < 1, we can consider the following dynamic optimization 
problem: 



inf J(c(-)) 

c{.)GC 



where the objective functional is given by 



-El 



U{cit))dt] + y^(0) 



which follows Belong Q. The state of Eq. (1231) can be reformulated as 

rT 



y{t) = e 



ac{s) + H\ ^Ty{u)x{u>o}<^{du) 



ds 



z{s)dWs 



where a is the uniform measure. Introduce the Hamiltonian function 



H{t,y{t),ys{t),z{t),c{t),p{t)) = -U{c{t)) + 



ac{t) +I3{jj^^ Ty{u)x{u>Q}{u)a{du) 



Pit)- 
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The associated adjoint equation satifies 

dp{t) = (^j^ (5p{s)-^ds^ dt, t G [0,r], 
. P{0) = 1- 

It follows Eq. ((24l) can be reduced to the following ordinary differential equation: 



(24) 



Pit) 



(3p{s)-ds, Pit) = -^p{t) 
s t 



which is solvable and by Theorem [321 we have the following result. 

Proposition 4.1. The optimal consumption is given by c{t) = {ap{t)) where p{t) satisfies Eg. d24D. 



5 Application II: Dynamic optimization of pension fund with delayed sur- 
plus 

In this section, let us turn to study Example 1.2 in Section 1. We will use the results obtained in Section 3 
to derive the optimal control. For simplicity, suppose g{-) is some linear function as follows 

g{y{t) - Ky{t - 6)) = ay{t) - any{t - 5) 

where a, /3 > 0. Then our model can be rewritten as 

' dy{t) = {{r - a)y{t) + \z{t) + aKy{t -5)- c{t)}dt + z{t)dW{t), t £ [0, T], 

y{t)=0, te[-6,0), (25) 

Denote the admissible control set by C = {c(-) G L|(0, T;R),t £ [0, T]}. It follows that if 5 is sufficiendy 
small, then Eq. (1251 ) admits a unique solution pair {y{-),z{-)). Introduce the objective functional of the fund 
manager as follows 



J(c(-)) =]E[ / Le- 
/o 



l-R 



-dt] -Kx{0), 



(26) 



with L and K are positive constants, p is a discount factor, and R € (0, 1) is index of risk aversion. 
The manager aims to maximize the expected objective functional by taking account both the cumulative 
consumption and initial reserve requitement. The optimal control problem is to maximize J(c( )) over C. 
The Hamiltonian function is given by 

Hit, y{t),y{t - S),c{t),pit)) = Le'^' ^ \_ + {(r - c^}y{t) + Xz{t) + aKy{t -5)- c{t)}p{t). 
The adjoint equation is 

' dp{t) = {{a - r)p{t) - aKE^'[p{t + 5)]}dt - Xp{t)dW{t), t G [0,T], 

p(0) = K, (27) 
[ p{t)=0, t£{T,T + 5]. 
Then from Corollary 13.31 we have the following result. 
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Proposition 5.1. If p{t) is the solution of ASDE H27]) , then the optimal consumption is given by c{t) = 

_i_ 

'■^^^—^ ^ and the optimal fund proportion in risky asset is 6(t) = where {y{t), z(t)) satisfies d25D . 
In the following, we aim to get the explicit solution of ASDE (|27] ). To this end, we first set 

M{t) = e/o -^'^^(^)-3/o^''i«, t£[0,T + 6]. 
It follows that M{t) is an exponential martingale and satisfies 

dM{t) = -XM{t)dW{t). 

Let p{t) = q{t)M{t), where q{t) is a deterministic function defined on [0, T + 6], then apply Ito formula to 
p{t), we have 

dp{t) = q'{t)M{t)dt - \q{t)M{t)dW{t), t G [0,r]. (28) 
On the other hand, substituting p{t) = q{t)M [t) into Eq.dJTl). we have 

dp{t) = {{a - r)q{t)M{t) - aKq{t + 6)E^'[M{t + 6)]}dt - \q{t)M{t)dW{t) 
= {{a - r)q{t)M{t) - aKq{t + 5)M{t)}dt - \q{t)M{t)dW{t), t G [0,T] 



Comparing (1281) and ( |29l ). if the following AODE has a solution 

' q'{t) = {a- r)q{t) - aKq{t + S), t G [0, T], 

< q{0)=K, (30) 
^ q{t)=0, te{T,T + 6], 

then p{t) = q{t)M{t) is a solution of ASDE The solution of AODE (gO]) can be obtained via the 
characteristic function as follows: q{t) = Ke^^ for t G [0,r], and q{t) = for t G (T, T + (^]. Here, h 
satisfies the following characteristic equation: 

h + ane^^ = (a - r). 

Note that a, r, k are the parameter of the state equation, so the above characteristic equation has solution 
h if the the delayed parameter 5 is small enough. In fact, denote F{h) = h + aKe^^ , then it follows that 

linift ^+cxi^(^) = +00. In addition, F'{h) > so F{h) is an increasing function of h, so there exists 

unique h such that F{h) = {a — r), thus q{t) as well as p{t) are uniquely determined. One remark to 
the parameter range. Set L = max{|a — r\,aK, A}, then we have the following parameter range to well- 
poseness of BSDE ^ and ASDE (|271): 

f 6L^S(l + 26^e) < 1, 

< (31) 

I AL'^6{l + 6'^e) + 6 < 1. 
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6 Application III: The dynamic optimization of linear delayed system 



Here, we revisit Example 1.3 of the backward system with time-delayed generator. The state equation is 
given by 

y(t)=C-^ [l3iy{s) + P2y{s-d)+jiz{s) + j2z{s-6) + av{s)]ds- z{s)dWs 

where a,/3i,/32 5 7i;72 are some constants, v{-) is the control process, and the class of admissible controls 
is denoted by Uad = G -^1(0; T; M), t G [0, T]}. The dynamic optimization problem is as follows: 

inf Jivi-)) 

where the objective functional is given by 

^ \ f-T ^ ^ 

+ Ky{0) 



j{v{-)) = Ie 



T 

R{t)v^{t)dt 







for some constant K and nonnegative function R{t) defined on [0,T]. By Corollary 13. 3[ the Hamiltonian 
function of our optimization problem becomes 

H{t,y{t),y{t-5),z{t),z{t-5),v{t)) 
= - ^R{t)v\t) + {av{t) + (3iy{t) + P2y{t - S) + -fiz{t) + j2z{t - S))p{t), 

and the adjoint equation becomes 

" dpit) = (-/3ip(t) - W + 5)]) dt + (-7ip(t) - 72lE^' W + 5)]) dW{t), t G [0, T], 
p(0) = K, (32) 
p(t) =0, t G {T,T + 5]. 

Similar to Application II, we can introduce the exponential martingale satisfying: 

dM{t) = jM{t)dW{t) 

where 7 is some coefficient to be determined, and set p{t) = q{t)M{t). Then we get the following AODE 
equation system: 

r q'{t)= -p^q{t)-l32q{t + 5), ^^^^ 

The first equation q{t) can be solved using the same method to Eq.(l30l). Based on it, we can plug q{t) into 
second equation to get the value of 7 thus the exponential martingale M (t) can be uniquely determined. 
Consequently, the optimal control is given by u{t) = where p{t) = q{t)M{t) is the solution of the 

ASDEi 
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